DOCUMENT RESUME 



ED 419 529 



IR 019 153 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Bowdish, Bruce ; Chauvin, Sheila; Vigh, Sandor 

Comparing Student Learning Outcomes in Hypermedia and Analog 

Assisted Lectures (HAL & AAL) . 

1998-04-00 

26p.; Paper presented at the Annual Meeting of the American 
Educational Research Association (San Diego, CA, April 
13-17, 1998) . 

Reports - Research (143) - - Speeches /Meeting Papers (150) 
MF01/PC02 Plus Postage . 

♦Academic Achievement; Curriculum Design; *Evaluation; 

Higher Education; *Hypermedia; *Instructional Design; 
Instructional Effectiveness ; * Instructional Materials ; 
Material Development; Medical Education; World Wide Web 
♦Analogical Imagery; ♦Digital Technology 



ABSTRACT 



The purpose of this paper is to report the results of a 
quasi-experimental study of the effects on student achievement resulting from 
the redesign of an existing anatomy course for a cohort of Occupational 
Therapy students. This study was based on a design strategy that combined two 
different instructional technology approaches to teaching and learning in 
lectures: (1) content -centered. World Wide Web-based hypermedia; and (2) 

situation-based, stand-alone hypermedia. The digital projection method of 
hypermedia-assisted lecture materials is described, which uses 
analog-assisted lecture materials within a conceptual framework incorporating 
"ecological psychology" and problem situations for learning. Findings are 
discussed in terms of content selection and presentation design guidelines 
derived from ecological theories of human learning. Also explored is the 
usefulness of this study for guiding future enhancements of medical education 
lecture materials. (AEF) 
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Introduction 

The“ primary purpose of this paper is to report the results of a quasi-experimental study of 
the effects on student achievement resulting from the redesign of an existing Anatomy course for 
a cohort of Occupational Therapy students. The paper also describes a digital projection method 
of hypermedia-assisted lecture materials (HAL) that incorporates analog-assisted lecture 
materials (AAL) (e.g., 35 mm slides, glass slides, and overhead materials) within a conceptual 
framework incorporating ecological psychology (J. Gibson, 1986; Shaw and Bransford, 1977; 
Shaw and Kinsella-Shaw, 1988; Shaw, Turvey and Mace, 1982; Turvey and Carello, 1981; 
Turvey and Shaw, 1979; Turvey, Shaw, Reed and Mace, 1981) and problem situations for 
learning (Hoffman and Richie, 1997; Lave, 1990; Spiro, and Jehng, 1990; and Young, 1993). A 
secondary purpose of the paper is to discuss the findings in terms of content selection and 
presentation design guidelines derived from ecological theories of human learning. Finally the 
paper explores the usefulness of this study for guiding future enhancements of medical education 
lecture materials. 

Perspectives 

Medical educators have been quick to embrace the use of computers in teaching medical 
students, particularly the use of hypermedia-based instruction (HAI). Yet, amidst the excitement 
and rapid developments of this form of instructional technology, evidence of its effectiveness is 
sparse (McLean, 1996). At present, the professional literature focuses on either the 
implementation of these technologies for use as self-paced, independent instruction or as 
supporting instructional activities, with little attention to how instructors might be.st utilize 
hypermedia in their lectures. 

This study wa.s based upon a design strategy that combined two different instructional 
technology approaches to teaching and learning in lectures: I) content-centered. World Wide 
Wch-based hypermedia; and 2 ) situation-ba.sed, stand-alone, hypermedia. Each approach differs 
in its target application and views of epistemology. For example, the Web-based hypermedia was 
ilcsignctl (dr seU-paced learning and devclopctl using a schema theory framev/ork. The HAL 
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materials developed for this study were designed primarily for group learning (although available 
over the campus network for self-paced learning after lecture) and were developed using a 
situated learning framework (Young, 1993). Despite their differing theoretical frameworks, this 
study holds to the premise that by carefully combining existing, Web-based, hypermedia with 
situation-based, stand-alone, hypermedia-assisted class presentations, effective, lectures (HAL) 
could be developed, implemented and tested. First, we will discuss the relevant literature related 
to schema theory, followed by a discussion of situated learning. Finally, we will discuss how the 
two theoretical perspectives influenced our design of effective HALs. 

Schema Theory Framework for Hypermedia Assisted Instructional Design 

The schema theory framework asserts that structured hypertext and hypermedia serve to 
facilitate learners assimilation and accommodation of new information into existing mental 
structures by providing a structured, interactive experience (Lawless and Brown, 1997). The 
information stored within HAI environments can be structured using a schema theory framework 
to form a dense or rich information space whose structure can be assimilated by learners who 
interact with it. Stated simply, the associations used to link nodes within the HAI will be 
perceived and encoded (learned) by those who interact with it. 

The schema theory framework acknowledges that HAI effectiveness may be influenced by 
learners’ “internal characteristics (e.g., prior knowledge, self efficacy, and interest) and different 
external constraints (e.g., learner control, instructional design and level of control)” - a view 
consistent with conventional perspectives of lectures (Lawless and Brown, p. 1 17). The control 
afforded learners in computerized environments refers to the extent to which learners direct the 
learning process. That is, learner control includes the selection and sequencing of content, display, 
pace, difficulty and (Merrill, 1975, 1984; Shyu and Brown, p. 86, 1992). Given the characteristics 
of the sample in the present study, our focus was more on the external constraints than on the 
internal characteristics. 

Keane, Norman and Vickers ( 199 1) argue that poorly designed hypermedia-based 
instruction may leave some individuals floundering within the very same information space in 
which they arc supposed to learn. The literature offers some suggestions as to how one might 
help lessen the severity of the con.sequences of poor learner-control. Tennyson and Rothen, 

( 1 979) describe methods for structuring hypermedia content, and techniques for including 
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opportunities for immediate feedback and advisement, but acknowledge the fact that in general. 




novice learners make poor instructional decisions. 

While the concept of learner-controlled instruction is not new to the use of computers as 
teaching aids for out-of-class or independent activities, few studies document the application of 
the emerging hypermedia technologies to large group instruction. Two studies used a subset of 
the schema theory framework. Cognitive Flexibility Theory (Spiro and Jehng, 1990), to design, 
implement, and test HAI in group and classroom settings. The first study (Spiro and Jehng, 1990) 
used the nonlinear aspects of HAI to control navigational features (level of control) to facilitate 
learners experiencing activities from multiple perspectives. They theorized that HAI could be 
structured so that, learners could view specific portions of the film Citizen Cane from various 
analytic perspectives. For example, after experiencing instructional materials that focused their 
attention a particular aspect of the film (e.g., film noire techniques) students’ viewed and 
discussed a preselected film segment (situation) in small-groups. Next, navigational features 
programmed into the HAI directed them to another section that focused their attention on a 
different aspect of the film (e.g., historical influences). Afterwards, students viewed the same film 
segment, and engaged in small-group discussions. This process was repeated so that over time, 
students were afforded multiple looks at an ill-defined problem situation - critically analyzing 



film. Spiro and Jehng (1990) proposed that solving ill-structured or ill-defined problems requires 
students to look at problems from multiple angles. They linked cognitively flexible thinking and 
perspective taking ability to effective problem solving (learning). 

A second study confirmed the use of a cognitive flexibility framework for the design of 
HAI, but implemented the instruction as a self-paced, stand-alone product within a computer 



classroom setting. The Cognition and Instruction Group at the University of Connecticut 
[CIGUC, 1995) applied the cognitive flexibility framework to the design of HAI named “Auntie 
EMM : an Electronic Mail Mentor” Their study demonstrated how navigational design features 
i^uidc students’ interactive learning experiences. Moreover, this study confirmed Spiro and 
Johng’s perspectives for solving ill-structured problems. Students in the “Auntie EMM” .study 
.Icmonslrated significant increases in their levels ot selt-ellicacy as well as their ability to use Rice 



Mail in a VM environment elTectivcIy. In each stiuly. the design ol IlAI centered on the 
importance of addressing learner-control and level of control. I3oth proposed that programming 
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the computer to limit students’ decisions while learning within the hypermedia environment is 
likely to be effective in small group and self-paced learning situations. 

Friedman (1996) suggests that implementing Web-based or technology-assisted instruction 
within the domain of medical education may prove potentially powerful, but it may also fail. He 
argues that, like technologies that were once innovative and are now standard tools (e.g., 35mm 
slide projectors, overhead transparencies, VCRs and TVs), determining the effectiveness of new 
technology will require the following: 1) integrating the use of new technology into the medical 
school curriculum; 2) using uniform standards to judge effective use, 3) implementing instruction 
in ways that exploit the problem-solving and visual aspects of the medium; 4) ensuring that 
students are tested the on material that was taught using the new technology: 5) updating and 
refining hypermedia-assisted instructional programs often; and 6) designing high quality 
hypermedia-assisted programs. Essentially, Friedman makes a strong argument that if Web-based 
instruction is designed and implemented poorly, the technology will likely undermine, rather than 
enhance, effective instruction. As discussed earlier, the known characteristics of poorly-designed, 

self-paced, hypermedia can be traced to problems associated with learner-control and level of 
control. 

Situated Learning Framework for Hypermedia Assisted Instructional Design 

Situated learning (Lave, 1990; Brown, Collins and DuGuid, 1989) is related to but not 
limited by the tenets of Problem-based Learning (PBL). That is to say that, problems can be 
construed as situations but not all situations must be construed as problems. Also, some 
proponents of PBL espouse schema theory frameworks of human learning. The situated learning 
framework (Young, 1993) differs from schema theory or an objectivist approach to instructional 
design in that it rests on the assumption that truth, like reality, is relative to the current situation 
or occurs naturally. Simply put, the situated learning framework assumes that there may well 
indeed be “more than one right answer.” In contrast, schema theoretic frameworks presuppose 
ab.solutc truth. As Young, Kulikowich and Barab (1997) state; 

"What becomes clear is the a.ssumption that concepts and principles exist 
indcpcridcntly from the students who know and use them. And the context in which they 
are applied is treated as incidental. This approach rests on an objectivist view that there is 
()ne truth to be known and learning is bringing each student’s understanding into line with 
this truth (p. I34j. *’ 
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According to Bridges (1992, p. 15), when compared with conventional, didactic, methods 
(lecture), Problem-Based Learning (PBL) or the problem-centered approach: 1) facilitates the 
development of “substantially more positive” attitudes toward the learning environment; 2) 
promotes studying for meaning rather than rote reproductions of instructor-proffered material; 3) 
promotes students’ abilities to complete instructional programs in less time and with fewer 
dropouts; 4) fosters steeper growth of students’ “basic knowledge of disciplines”; and 5) develops 
clinically competent behaviors. 

Young (1993) proposed that “technology-rich” situations such as the “Auntie EMM” and 
“Citizen Kane” studies can be designed to facilitate learning (p. 48). In his article “Instructional 
Design for Situated Learning” (1993), he argues that “ideas of ecological psychology provide the 
background for describing four broad tasks for the design of situated learning: selecting the 
situations, providing scaffolding, determining and supporting the role of the teacher, and assessing 
situated learning” (p. 43). Hoffman and Ritchie (1997, p. 97) suggest that interactive multimedia 
can address some of the problems typically associated with problem-based learning, such as 
relying too heavily on written cases, or verbal or written vignettes; ignoring students’ 
competition for high grades and test scores; and discounting students initial discomfort with the 
increased degrees of freedom experienced in a PBL environment. 

Barab, Bowdish, Young and Owen, (1997) propose that the interaction of student and 
HAI functions as a dyad. And like the apprentice situations discussed by Lave (1990), HAI can 
be designed to facilitate cognitive apprenticeships by externally representing the thinking of the 
HAI developers. They suggest that the discourse and co-construction process obviated by the 
interaction between students and HAI require designers to incorporate aspects of authentic 
environments to maintain what amounts to an ecological validity (Young, Kulikowich and Barab, 

1997, Young, 1993, Young and McNeese, 1995). 

Situating learning scaffolds students. When they adopt the goals posed within realistic 
problem-solving contexts, the net effect is a constraint to, or specification of, their intentions 
(Barab and Young. 1996). This specification helps learners make sen.se of the situation. As the 
informational fabric of the problem context collides with the learners’ intentions, their interplay 
.serves to obviate probable patterns or potentials lor action (Sliaw, I urvey and Hawkins 
I.ahoralories, 1 9H 1 ). 'I’lius. problem sitiiatioiis and problem-sol ving environments can be 
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combined to facilitate interactions where knowledgeable behavior is literally co-determined as the 
properties of the student and hypermedia dynamically interact (including those of other students 
and/or additional sources of HAI) (Barab and Young, 1996). 

Admittedly, the PBL approach in medical education does result in slightly lower 
performance on standardized exams (e.g., USMLE, Step I) than for students enrolled in 
traditional curricula (Friedman, de Blick, Sheps, Greer, Mennin, Norman, Woodward, and 
Swanson, 1992). However, some argue that such performance data are the result of the objective 
truth-based scoring mbric that emphasizes the number of “right” responses recorded on multiple- 
choice and short essay tests (Young, Kulikowich and Barab, 1997). This seems to confirm 
Friedman’s (1996) suspicion that special consideration must be given to ensure that testing 
formats reflect accurately the content and context of teaching and learning using the new 
technology. If PBL centers on the entire situation, (group dynamics and problem-solving 
included), and traditional testing only measures students’ declarative, procedural and conditional 
knowledge (Alexander, 1992), then hypermedia assisted lectures (HALs) may well represerit an 
inclusive midpoint. That is, HALs seem to have the potential for facilitating students’ learning 
specific content and skills (schema theory perspective) within authentic situations or problems 
(situated-learning perspective). Our belief that the use of HALs can provide instmctors with a 
model for scaffolding students’ learning by controlling both what is available for students’ learning 
and the situations in which they learn guided the development of research questions for this study. 

Research Questions 

In light of the previous discussion, we posit the following research questions.' 

1 . Will the use of HALs in a gross anatomy course produce significant differences 
over time between or among treatment and control cohorts with respect to 
participants anatomy knowledge and laboratory practical examination scores? 

2. Can participants’ performance profiles (as measured by their anatomy knowledge 
and laboratory practical examination scores across three different occasions) be 
accurately classified into different treatment groups? 

2a. If so. which course block (occasion) measures demonstrate the best fit with the 
cli.scriminant function? 




Mellioil 
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The study was conducted in a private, southeastern United States medical school. The 
sample consisted of five cohorts (N = 162) of occupational therapy students enrolled across a 
penod of five consecutive years (1993 - 1997) in a gross anatomy course. Three of the cohorts’ 
archived data (1993-1995) were obtained and analyzed to control for selection bias (Cook and 
Campbell, 1979, p. 132). Analysis of each cohorts’ group mean revealed that group performance 
on each of the dependent measures was consistent (see Table 1). The remaining two cohorts 
comprised the control group (1996, n = 30), students that completed the couree following the 
traditional AAL format, and the treatment group (1997, n = 30), students that completed the 



course using the new HAL format. 

Table 1: Achievement Scores Calculated Using Archived Cohort Data (1993 - 1996) 



— — Group Examination Mean .<5rnre 


1993 Cohort (n = 30) 


85 


1994 Cohort (n = 37) 


85 


1995 Cohort (n = 36) 


84 


1 996 Cohort ( n= 30) 


87 



Analysis consisted of a 2X3X2 Between Groups Repeated-Measure, Multivariate Analysis 
of Variance (RMANOVA). Independent variables were Treatment methodology: 1, lecture type 
(control -AAL and treatment - HAL); and time (block-I, block-2 or block-3). Dependent 
measures were participants' objective/knowledge exam scores (Obj 1, OBJ2 and OBJ3) and 
laboratory practical exam scores (Subj 1 , Subj2 and Subj3). A post-hoc discriminate function 
analysis (DFA) was conducted to investigate significant main effects to determine the degree and 
nature of each dependent variable's contribution to the composite (Tabachnick and Fidell, 1989). 

MATERIALS 

The content projected in lecture (HAL) can be understood best as a mix of existing web- 
based hypermedia and in-class hypermedia assisted presentations generated using Microsoft 
PowerPoint version 4.0. Both components (web-based and in class) were carefully integrated to 
yield the HAL series. Of .special note is the fact that HALs facilitated the integration of 
instructional materials that were generated by medical faculty from various di.sciplincs (horizontal 
integrationj as well as medical faculty from within the di.scipline of anatomy from both basic 
.science and clinical perspectives (vertical integration). 



Prior to the generation of HAL materials, the existing course (anatomy for Occupational 
Therapy students, AAL) was carefully analyzed to establish available components suitable for 
control within a quasi-experimental, cohort research design for formal institutions (Cook and 
Campbell, 1979). Once analyzed, we implemented controls to ensure that external threats like 
selection bias, and internal threats such as the course content (handouts, dissection materials and 
textbook), assessment methodologies (e.g., knowledge exams and laboratory practicals), and 
instructional time schedules were identical for both groups. Our intent was to analyze instructional 
situations where the control group cohort experienced AAL, and the treatment group cohort 
experienced HAL. 

With design controls in mind, we developed the interactive hypermedia presentations for 
classroom presentation purposes in accordance with Young’s (1993) guidelines for the design of 
instructional materials that included the following: a) incorporate situations that afford the 
acquisition of material that the teacher (or designer) wishes each student to procure; b) to provide 
guidance or “scaffolding” for novices within the learning situation in a manner which also aUows 
experts to operate; c) to include supporting features that allow one to track learners’ progress as 
they access distributed information and/or works collaboratively with others thereby facilitating 

the skills of the designer or teacher; and d) a defined role and method of assessing learning within 
the presented situation, (p. 46) 

We began by addressing the classroom situation itself, namely, when lecture begins. 
Generally, lectures begin with someone stepping up to a podium or assuming a position in front of 
the class and announcing that they are about to begin. We immediately thought that we could 
develop an opening screen which communicated the course objectives using a multimedia display, 
(see Figure la). The result was a screen that included the basic information in text (i.e., 
instructors name, school name, course name etc.) and more general information using graphics, 

sound and a Quicktime™ movie generated from the anatomical cross-sections from the Visible 
Human Project. 

Each lecture presentation design began with a review of the objectives retained from the 
analog-assisted lectures (AAL) that were used with the 1996 cohort (control group). Media 
•selection was determined by analyzing the conceptual aspects of the subject matter. For example, 
lor concepts sucli as .systems, a .scliematic drawing was a starting point to simplify and label 
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components of the system for subsequent investigation and instruction pertaining to the 
relationships between structure and function. We decided that apprenticing students to functional 
relationships by using schematics, illustrations, radiologic imagery, photorealistic imagery, digital 
video, and eventually the cadaver in the laboratory, would provide students with multiple 
horizontal perspectives of the lecture content (Spiro and Jehng, 1990). Additionally, relationships 
that described the level of analysis being displayed using the image (micro to macro) and methods 
for determining one’s orientation between dr within analytic levels were included graphically (see 
Figure 2). To accomplish this, images were juxtaposed, overlaid-lines connecting related images 
were added, and thumbnail images pertaining to previously viewed analytic levels were included. 
The idea was to leave obvious footprints in various ways so that students could literally see how 
the instructor (subject matter expert) used the information being displayed. Figure 3 provides an 
example of a series of screen displays used in the HALs. Furthermore, as problem-situations were 
presented in the form of clinical cases (Hoffman and Ritchie, 1997), students would be afforded 
opportunities to see how the instructor vertically integrated the basic science knowledge 
presented in the course (including the supporting HAI). 

During this phase of the instructional design, presentation features pertaining to 
background color, font size, font type and generic navigational features (level of control) were 
standardized. Our goal was to create displays that were easy to see, consistent and easy to 
duplicate. Standardizing the “look and feel” of hypermedia materials (web-based and in-class) 
reduced production time substantially. Also, by designing one curricular component at a time, we 
were able to reflect upon our experiences and improve each module sequentially. 

Various examples from each of the aforementioned horizontal and vertical perspectives 
were selected and digitized. This phase of the production process proved incredibly time 
consuming and detail ridden. However, we did learn that if done properly, digitized images could 
be re-purposed for use in several different contexts (e.g., printing, web display, texture maps, and 
inclusion in digital movies) and greater coherence emerged regarding the situations we planned to 
display. Overall, 500 new line art and photorealistic images and five Quicktime™ movies were 
added to the 200 images previously used in the AAL version ol the course. Each image was 
digitally optimized and tested to ensure that it would project without being washed out by the 
projection process. This pha.se of the deveh^pinenl/production process proved exceptionally 
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important as some images did tend to look different when viewed on a computer screen when 
contrasted with the projected image. 

Near the end of our initial development phases (i.e., lectures for the first block) we 
realized that we hadn’t addressed the end of lecture as a situation. We decided to take common 
elements from the opening display and convert them to yield a closing display (see Figure lb). 
Closing displays consisted of a review of the behavioral objectives for the lecture displayed with 
text and a Quicktime™ movie comprised of selected graphics and photorealistic images presented 
in the lecture. Our intent was to provide an engaging summary of the lecture in a format that was 
easily assembled and accessed. 

Once a block of HALs was completed, we revisited the level of control issues that resulted 
from our navigational controls and role played a prototypic instructional situation. We decided to 
create an index of links within PowerPoint to allow instructors and students to jump between 
lectures quickly and easily to promote the opportunities for classroom interactivity. We also 
decided to make network resident, digital copies available to students in the form of Adobe 
Acrobat version 3.0 Portable Document Files (PDF) for viewing after HALs had been presented. 
Students’ responses to both the PDF and Index features of HALs were very positive. However, 
placing numerous image-laiden PDF files on the campus network did prompt a fair degree of 
concern on the part of our network administrators. 

Once we finished the first block, we moved on to the second and third. In each instance, 
we uncovered seemingly small details associated with the design. Once again, we reflected on our 
previous efforts and revised where necessary. In the end, we learned that updating HALs can be 
done fairly rapidly. Their consistent structure and index-based accessibility made it easy to find 
and update lecture content — a key piece of the argument proposed by Friedman (1996). 

Results 

One participant attrited from the treatment group, therefore data were obtained from 59 
participants. The total cohort comparison sample entered into the RMANOVA consisted of 75% 
females and 25% males. Prior to beginning data analysis, Windows95-ba.sed SPSS version 7.5 
(SPSS Inc., 1996) was u.sed to test the data .set against the statistical assumptions appropriate to 
RMANOVA and DFA. 
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Descriptive statistics reveal that the treatment cohort’s (63%) group mean score on 
laboratory practical examinations were noticeably lower for the first block as compared to the 
control cohort’s (85%) group mean score (see Table 2: Cohort groups’ means and standard 
deviations). The treatment cohort’s group mean score surpassed the control cohort’s group mean 
score for the second block (85% and 83% respectively) but by the end of the third block, their 
group mean scores were identical (82% and 82% respectively). 
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Table 2: Cohort Groups’ Means and Standard Deviations 
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GROUP MEAN 


OBJl 


OBJ2 


OBJ3 


r SUBJl 


SUBJ2 


SUBJ3 


Control 


.82 


.95 


.89 


.85 


.83 


.82 


Treatment 


.80 


.95 


.97 


.63 


.85 


.82 


Total 


.81 


.95 


.93 


.74 


.84 


.82 


GROUP SD 


1 DBJl 


n DBJ2 


OBJ3 


^UBJl 


rSUBJ2 


SUBJ3 


Control 


.075 


.039 


.054 


.098 


.090 


.087 


Treatment 


.079 


.029 


.034 


.115 


.095 


.105 


Total 


.077 


.034 


.059 


.154 


.092 


.096 



Results obtained from RMANOVA with Wilks criterion, revealed that the combined DVs 
were stgmficantly affected by lecture type for both within-subject, F (4, 226) = 26.58, p < .001, 
and between-subject effects, F (2, 56) = 13.95, p < .001. The msults reflected a strong association 
between lecture format and the combined DVs across occasions,r,= = .54, and within occasion, r,^ 

= .33 (see Table 3; RMANOVA for cohort groups’ objective and laboratory examination scores 
across three instructional blocks). 

A post-hoc, step-wise discriminant function analysis (DFA) was conducted (see Table 4: 
Classification Results, Table 5: Stepwise DFA Statistics and Table 6: Summary of Canonical - 
Discnminant Functions) to determine whether participants' performance profiles could be useful 
to classify future cases into lecture-type groups and to evaluate which block measures 
demonstrated a best-fit with the statistical function (Tabachnick and Fidell, p.479, 1989). 

Results revealed that participants' profiles could improve prediction by 91% (adjusted with 
Cohen's Kappa) over and above chance (Cohen, p 29, 1978). Pooled within-groups correlations 
between discriminating variables and canonical discriminant functions revealed that participants’ 
first-block laboratory practical exam scores (r = .73) and third-block knowledge exam scores (r = 

- .58) demonstrated the best-fit with the derived function followed by participants’ third-block 
laboratory practical exam .scores (r = .18), first-block knowledge exam .scores (r= .13), .second- 

block knowledge exam .scores (r = -.10). and second-block laboratory practical exam scores (r = 
•OS). 
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Table 3: RMANOVA for Cohort Groups’ Objective and Laboratory Examination Scores 
Across Three Instructional Blocks 

Omnibus Effect 



F (4, 226) = 29.19; p < .001 Effect Size = .50; Noncent 229.5; Power 1.00 



Effect DF 

Group by Block 4,54 

Block 4,54 


F Significance 

29.18 .000 
60.33 .000 


Effect Size Noncent 
.68 116.57 

.82 241.32 


Power 

1.00 

1.00 


Table 4: Classification Results 




Predicted Group Membership 


Actual Group 


No. of Cases 


Control 


Treatment 


Control Cohort Group 


30 


28 

93.3% 


2 

6.7% 


Treatment Cohort Group 


29 


2 

6.9% 


27 

93.1% 



Percent ot "grouped" cases correctly classified: 93.22% 
Cohens Kappa adjustment for chance probability: 91% 



Table 5; Stepwise DFA Statistics 



Wilks’ Lambda 



Step 


i Entered 


Statistic 


dfl 


df2 


df3 




Exact F 




Statistic 


dfl 


df2 


Sig ■ 


1 


Subjl 


.469 


1 


1 


57 


64.44 


1 


57 


.000 


2 


Obi3 


.320 


2 


1 


57 


59.49 


2 


56 


.000 


Note: 


a - Minimum partial F to enter is 3.84 












b - Maximum partial F to remove is 2.71 












Table 6: Summary of Canonical Discriminant Function 












Function 


Eiffen Value % of Variance 


Cumulative % Canonical Correlation 


1 


2.124 


100 


100 




.825 





Standardized Canonical Di.scriminant Equation 

D = -.69(Objective Examination Block 3 Score) + .82(Lab Practical Block 1 Score) 



Unstandarclizod Canonical Di.scriminant Equation 

D = 8.34 - 1 5. 1 7(Objcctivc Examination Block 3 Score)+ 7.67(Lab Practical Block 1 Score) 
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Discussion 

Our study was guided by the three research questions posed on page 8 of this paper. 

Each is addressed in terms of our findings in the paragraphs that follow. 

Will the use ofHALs in a gross anatomy course produce significant differences over time 
between or among treatment and control cohorts with respect to participants' anatomy 
knowledge and laboratory practical examination scores? The results revealed that at the end of 
the course, participants’ cumulative performances on objective/knowledge examinations were 
significantly higher for participants taught with HAL (Mean = 90.3%) than with AAL (Mean = 
88.7%). However, participants’ cumulative laboratory practical examination scores revealed a 
significantly opposite pattern (HAL Mean = 76.3 and AAL Mean = 83.3). We submit that lower 
laboratory performance scores are a direct result of students’ stniggle to adapt to the redefining 
of their roles as learners in lecture during the first block of the course. The fact that they are 
equally able to apply their knowledge in the laboratory in both the second and third block seems 
to suggest that instructors might consider increasing the number of examinations to guard against 
mathematically penalizing their students as they adjust to their new roles. 

Second, the doubly-multivariate RMANOVA design implemented in this study helped to 
measure participants’ performance on knowledge and laboratory practical examinations in a 
manner that accommodated time both statistically and practically (i.e., provided time for 
participants’ role redefinition). Careful analysis of the profile of participants' performance on 
exams (both objective and practical) throughout the course suggested that Pina and Savenye’s 
(1992) argument regarding changes in roles warrants further investigation. The data obtained for 
this study revealed that over time, participants’ block scores for each of the dependent measures 
described significantly different performance patterns. For example, it was hypothesized that 
HAL participants’ performance measures would suffer initially (block one) due to changes in 
expectations of students’ roles as learners that are often associated with situated learning 
environments. Indeed, participants from the treatment cohort (HAL) did exhibit significantly 
lower mean performance scores for the first block (HAL 79. 63 and AAL 82, 85; but, they did 
change signilicaiuly by the end of the .second (HAL 95.84 and AAL 95, 83; and third blocks 
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(HAL 97, 82 and AAL 89, 82). The results here suggest that students adjusted to new roles and 
expectations and ultimately enhanced their learning and performance on course measures. 

Third, the conceptual framework that was used to design the HAL series predicted that 
participants' ability to apply the content would be enhanced because the HAL format incorporated 
principles of situated learning and cognitive flexibility. While a tendency might be to review only 
cumulative results of students’ performance on course measures, examining student learning 
outcomes through the use of performance profiles provides a much richer and accurate 
description of the influence of instructional innovations such as ELALs. 

Can participants’ performance profiles (as measured by their anatomy knowledge and 
laboratory practical examination scores across three differerit occasions) be accurately 
classified into different treatment groups? Results of the RMANOVA and discriminate function 
analysis (DFA) demonstrate clearly the power of repeated measures to detect statistically 
significant differences in students’ performance profiles. The accuracy of the DFA (91%), after 
adjusting for chance probability (Cohen’s Kappa, 1978) supports strongly the effectiveness of 
HAL for enhancing students’ learning outcomes. Methodologically, the RMANOVA and DFA 
are useful tools for detecting levels of effectiveness for instructional innovation such as HAL, 
especially in situations when students are organized as learning cohorts. 

If so, which course block (occasion) measures demonstrate the best fit with the 
discriminant function? Students’ performance on the first block laboratory practical examination 
represented the best fit with the discriminant function. This result reinforces the notion that 
students’ experience changes in their role perceptions and need time to adjust. The only other 
measure to enter the stepwise DFA was the third block (occasion) knowledge examination score. 
These results reinforce the degree of learning enhancement expected in using HALs. Also, the 
findings here lend support to the notion that students need clarity about role expectations and 
specific encouragement to assume active learning roles early in the course. Clearly, the results of 
this study offer a model for examining the effectiveness of instructional technology and raise new 
questions about the nature of learning environments that exist and may be found in lectures - 
traditional and hypermedia-assisted. For example, to what extent are instructors and students, 



individually and collectively, responsible for facilitating effective teaching and learning proces.scs? 




I U)w do instructor actions influence stiulerit involvement and vice versa? 
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hypermedia encourage or constraint interactions and active involvement - within class and 
outside of class? Does HAL extend hypermedia in a way that meaningfully enhances learning 
environments beyond the traditional classroom walls? 

Conclusions and Future Directions 

Our design framework (HAL) posits that an intermediary (the instmctor) can generate 
appropriate matches between problems (i.e., clinical cases) and problem-solving environments 
(e.g., HAI) by linking or integrating lecture materials with existing Web-based hypermedia. 
Through careful content organization (as detailed by schema theory) and high-fidelity problem 
representation (Young, 1993), HALs can be developed in such a way that the instructor can 
effectively broker learner-control and level of control within a traditional lecture setting. We 
believe that careful use of the multiple perspective-taking techniques described by Spiro and 
Jehng, (1990) combined with the situational design guidelines of Young (1993) can provide 
scaffolding, determine and support the role of the teacher, and utilize outcome-based assessments 
of the learning environment (situation) in a manner which takes into account achievement - - v 
measures. The HAL approach capitalizes on the idea that instructors can facilitate student 
interactions and active involvement in lecture settings. Ln many ways, expectations for student 
interaction and active involvement have been reserved for small groups and PBL formats. The 
sequencing of content, and how and when it is displayed in HALs can provide effective scaffolds 
upon which the raison d’etre of teaching and learning are made obvious to the learner. Pina and 
Savenye (1992) submit that much of the danger associated with PBL can be traced to significant 
changes in the teacher's role due to the increase in learner-control in teaching and learning 
processes. The HAL approach affords both students and instmctors a mediated environment 
where they can gradually adjust to the new format and roles prompted by HAL. The nonlinear 
structure, information, and control embedded within HAL increase the degrees of freedom 
available for instructors and students compared to AAL. The difference is that the situated 
learning principles incorporated in HALs help students understand the relevance of course content 
information because it is presented within realistic situations. 

We are prompted to reflect on Friedman’s requirement that new technologies must be 
integrated into the curriculum in a way that high quality, up-to-date, hypermedia-assisted 
materials arc used, and managed realistically, in teaching learning, and assessment. That is, the 
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content and context for teaching and learning are reflected in the same way for testing what 
students have achieved. Friedman argues that this also includes authoring programs. In this 
study, our instructional design was based upon the idea that presentation software (e.g., Microsoft 
PowerPoint 4.0) can be used to generate what amounts to a midpoint between two relatively well- 
known extremes: 1) the detailed programming requirements of authoring self-paced HAI; and 2) 
the relative ease of using interactive multimedia to generate situated learning materials (e.g., 
HALs). Therefore, new developments in presentation software (e.g, . Microsoft PowerPoint 4.0) 
offer realistic ways for interested instructors to manage teaching, learning, and testing with 
hypermedia. 

Conclusions drawn from this study may be useful to other medical schools contemplating 
the development or re-purposing of lecture materials and methodologies. Additionally, the 
findings and conclusions of this study seem to be applicable to a wide variety of higher education 
settings. The use of specific elements of content selection and presentation design strategies 
derived from ecological theories of human learning (Young, 1993) and quasi-experimental, 
cohort-based research designs (Cook and Campbell, 1979) hold promise for action-oriented 
research and assessment within formal educational institutions. It also suggests a need to both 
replicate and extend the study on a larger scale to examine its effectiveness across institutional 
settings, content domains, and delivery mechanisms (e.g., distance learning). 
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